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expected, the univariate Box-Jenkins method produced short term 
forecasts which were superior to those of the least squares regression 
models. However, the regression forecasts were highly accurate and were 
considerably less expensive to obtain. Only one multivariate Box- 
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ABSTRACT 


This thesis compares three types of models developed to predict overhead costs 
for seven government aerospace contractors. The methodologies utilized to develop 
the models include generalized least squares, univariate Box-Jenkins, and muitivariate 
Box-Jenkins procedures. The results of those models are compared using three 
nieasures of effectiveness: correlation coefficient between actual and predicted values, 
root mean squared error divided by the mean of the actuals, and mean absolute 
percentage error (in percent). As was expected, the univariate Box-Jenkins method 
nroduced short term forecasts which were superior to those of the least squares 
regression models. However, the regression forecasts were highly accurate and were 
considerably less expensive to obtain. Only one multivariate Box-Jenkins model could 
be developed. The results of this model were marginally superior to the related 
regression model and significantly inferior to the univariate Box-Jenkins model for the 
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I. INTRODUCTION 


Overhead costs often constitute a large portion of total product costs in many 
industries. In the case of government aerospace contractors, overhead costs may 
comprise as much as fifty percent of the contract cost negotiated by the Federal 
government. Historically, overhead costs have been predicted by applying estimated 
overhead rates to estimated labor hours in several categories of operations. Tnese 
category totals are summed to arrive at a total overhead cost for a particular product. 
This procedure of estimating overhead is greatly affected by changes in the level of 
operations. Fluctuating production rates often lead to overhead rates which display a 
lagged relationship between the applied rates and actual overhead costs. Consequently, 
this particular method of predicting overhead costs produces estimates which are 
inadequate. 

Alternative methods for predicting overhead have been proposed. In many cases, 
these methods attempt to establish a direct relationship between costs and factors 
related to direct production. Thus the need for reliance on estimated overhead Tatecme 
eliminated. Various least squares regression models have been proposed in attempts to 
establish the desired direct relationship. One such model was developed with simplicity 
in application as a strong consideration [Ref. 1: p.7]. This allowed users of varying 
degrees of statistical familaritv to apply the model in actual work conditions. This 
model intentionally utilized the minimum number of explanatory variables necessarv to 
achieve accurate results. Several applicable independent variables were considered and 
observations of direct personnel demonstrated the strongest relationship with overhead 
costs [Ref. I: p. 20]. As with any economic trend data, autocorrelation must be 
suspected. The use of quarterly observations in the model required testing for first 
order AR(1) and fourth order AR(4) autoregressive processes. Appropriate tests and 
corrections Were incorporated into the model to ensure the absence of bias in the 
standard errors of the coefficients and in the R-squared statistic [Ref. 2: p.283]. Tne 
resuits of this model are included for comparison purposes. 

\ second alternative to the use of estimated overhead rates is the Box-Jenkins 
method of forecasting. This method, which is designed to produce highly accurate 


short term forecasts, allows for a wide range of possible models to apply to a particular 


economic series [Ref. 3: p.I1]. The degree of statistical sophistication required to apply 
this method 1s far greater than that required for the regression model mentioned above. 
In addition, the use of the Box-Jenkins forecasting method requires far greater 
computer resources during the three stages of identification, estimation, and forecasting 
than mest regression models. Consequently, the Box-Jenkins method has not been 
applied as a forecasting tool in many cases in which it would be a logical alternative. 

The Box-Jenkins transfer function (multivariate Box-Jenkins method) utilizes 
deviations from appropriate means of an input (X) and of an output (Y) to establish a 
relationship from which forecasts can be made. The connecting tool between these two 
series is a linear differential equation. This extremely complicated forecasting tool 
Fequires €Xtemsive expertise om the part of the statistician. Once again, this highly 
effective forecasting tool has been underutilized. 

This paper attempts to develop usable least squares regression, Box-Jenkins, and 
Box-Jenkins transfer function forecasting models for seven government aerospace 
contractors. The effectiveness of each model is measured by withholding the last four 
cata observations during the model development phase and using the model to forecast 
the withheld observations. The deviation of the actuals from the predicted values have 
been indicated with three measures of effectiveness: correlation coefficient between 
actual! and predicted values, root mean squared error divided by the mean of the 
Metuals, aadanean absolute percentage error (im percent.) Ihe development and results 


of each model are included for comparison purposes. 


If. DATA 


The data have been supplied bv seven government aerospace COntlactorsaames 
maintain confidentiality, all references to specific contractors will be with the lapere: 
through G. A specific reporting format was utilized by each contractor during data 
collection. Quarterly data spanning the period beginning in the first quarter of 1979 
and continuing through the third quarter of 1986 was requested from each of the seven 
contractors. Usable data were obtained from each contractor for large portions of the 
requested period. However, only one contractor was able to supply all thirty-one 
observations. 

In the reporting format, overhead costs were composed of costs from three major 
categories which had been further refined into six subcategories. The first major 
category, labor related costs, was composed of two subcategories: indirect salaries and 
fringe benefits. The second major category was facilitv costs. The third major 
category is composed of three subcategories: electronic data processing costs, 
independent research and development and bid and proposal costs, and all other 
overhead costs. 

All components of overhead costs were converted to constant 1984 fourth 
quarter dollars. The labor related cost categories adjustment was accomplished 
through the application of the Bureau of Labor Statistics SIC 372 price index for 
production worker average hourly wages for the aircraft and parts industry. In the 
case of this index, monthly indices were averaged to produce quarterly indices. Gross 
National Product Deflater indices published by the Bureau of Economic Analvsis were 
applied to the two remaining major categories. Facility costs were adjusted with the 
GNPD gross private domestic fixed nonresidential investment index. The GNPD 
personal consumption services index was utilized to adjust the fina! major category. As 
with all indices, those used were imperfect. They were chosen because they provide the 
best adjustments for inflation among all readily available and relevant indices. 

Data pertaining to direct production were obtained from each contractor. The 
only direct production data set utilized in the analysis was direct labor personnel. This 


category of data did not require adjustment for inflation. 


> a, 


WI. VMODEL DEVELOPMENT 


A. THE REGRESSION MODEL 

The model utilizec in this project was designed at the request of the Naval Air 
Svstems Command as part of its contractor overhead tracking project. The model 
capitalized on the explanatorv and prediction properties of least squares theory. 
Individual regression models were developed for each contractor in an effort to 
accurately predict future overhead costs. Simplicitv in application has been a key 
factor in the model’s development. 

The application of least squares to economic trend data almost immediately 
implies autocorrelation in the error terms of the regression. With the presence of 
autocorrelation, the estimates of the coefficients are unbiased and consistent. 
Rietever, they are mot efficient. Ihe estimate for the variance of the coefficients are 
biased. Positively autocorrelated errors produce a coefficient variance which is 
underestimated because of a downward bias in the estimate of the variance. The 
downward bias will produce a confidence interval which 1s narrower than it should be 
ioimeaten coeiicient. For this reason, tests of the null hypothesis that the coefficient is 
equal to 0 will be rejected in instances in which it should be accepted. Likewise, 
autocorrelated errors will cause exaggerated R* and F statistics when an ordinary least 
squares model 1s applied. 

The effects of autocorrelation can be eliminated through the application of 
generalized least squares (GLS) procedures. The application of GLS to autocorrelated 
data produces estimators of the coefficients which possess the properties of maximum 
likelihood estimators. Therefore, the GLS estimators of the regression coefficients and 
the variances of these coefficients will be unbiased, consistent, and asymptotically 
efficient. This will lead to more reliable estimates of R* and F. [Ref. 4: pp. 302-311] 

This project examined quarterly data. For this reason first order AR(1) and 
fourtn order AR(4) autoregressive processes are suspected. AR({1) processes are 
cetectable with a Durbin Watson Test. In those cases where AR(1) is present, the data 


has been transformed in the following manner: 
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e, = residuals from the OLS regression. 


In equation 3.2, cn defined in this manner is the two stage Prais-Winsten 
estimator derived by Park and Mitchell [Ref 5]. The AR(4) process detected in the 


model, a special form of the general AR(4) process, 1s written as 


E. = Pye, 4+ Uy. (eqn 3.3) 


The general form for AR(4) processes is 


“\ IN JN aN 
E,= PpErp 7 P2Er.2 FT PZEr.3 7 Pgerg F Uy. 
(eqn 3.4) 


In both cases 
z y 
D> Noe: 
In this model, the effects of the three prior quarters are assumed to be negligible 
while the effect of the quarter one vear previous is considered significant. This special 


form of AR(4) process is detectable with a Wallis Test [Ref. 6]. If AR(4) processes 


were detected, the data sets were transformed in the following manner 
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parameters to be estimated, and dy is the Wallis Test Statistic written as 
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dy = ——-_ (t= 12...) (eqn 3.8) 


where 


e, = residuals from the OLS regression model. 


This particular estimator, derived by Theil and Nagar [Ref. 7: p. 287], was found 
to be the most efficient among nine alternative estimaters applied directly to these data 
seus, | Ret. S: 5 49] 

After each OLS model was computed, the Wallis and the Durbin Watson 
Statistics were examined for the presence of AR(4) or AR(1) processes. Appropriate 
transformations were made and the models were reestimated. aAt the conclusion of the 
transformation and reestimation phase, all traces of autocorrelation had been removed 
from the GLS models. In each case, the residuals were examined and all tests for 
normality were accepted. 

The following procedure was applied to each of the seven contractors. A detailed 
Pmieseliaiion Of the results will be made for contractor A. The results from the 


remaining contractors will be made in a summarized fashion with appropriate 


comments. Direct comparison of the regression models between two contractors does 
not indicate relative efficiency. Aside from the organizational differences within each 
firm. each of these contractors specializes in a particular branch of the aerospace 
industry. Examples include fixed wing aircraft and aircraft engine producers. For 
these reasons, comparison of the models between contractors is inappropriate. 
Contractor A supplied data spanning twenty-six quarters. Therefore, all three 
models: regression, Box-Jenkins, and Box-Jenkins transfer function, are based on 
twenty-two observations. The remaining four observations are withheld during the 
model development stage and are used for comparison purposes with predicted values 
during the forecasting stage. The results of the OLS and GLS models are presented in 
Table 1. The adjusted R? value of .1828 and the F value of 5.93 indicate that the OLS 
model is poor. The Wallis Statistic of .5801 indicates the presense of the special form 
of AR(4) suspected in the data. The appropriate transformation to correct for AR(4) 
was made and the model improved significantly: adjusted R* =.8718 and F= 150.6569. 
At this point, the residuals indicated the effects of first order autocorrelation. A 
second transformation was made and the model improved slightly: adjusted R- = 8787 
and F= 160.5558. Both forms of autocorrelation had been completely removed at the 


completion of the second transformation. 





TABee 4 
REGRESSION MODEES FOR COAG es 


OES GLS AR(4) ~GLS AR(1) 


Adjusted R? 183 872 879 

Po Statisnic 5921 150.657 160.336 

Intercept: $4778.277 11487.623 3047.305 
| Standard Error: 68215.1452 3924.463 2138.97 
| Slope: 13.960 15.701 15.946 
! Standard Error: Seon 1.279 1.259 
| Durbin Watson Statistic: Pos 644 1.99] 
| Wallis Statistic: 580 1.351 1.432 
| Estimate of py 187 694 080 
| Estimate of py ie 334 294 
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The regression models for the remaining six contractors are depicted in Table 2. 
A single transformation was required to eliminate dll indications of autocorrelation in 
the cases Oi Contractors B and C. The required transformation removed the effects of 
ie yeprccess. Ine Gata Supplied by Contractor — was free of the effects of 
autocorrelation and the model listed in Table 2 is the OLS model developed for this 
contractor. The remaining contractor data sets Were transformec twice to achieve the 
desired residual characteristics. Ail of the models derived during this stage possess 
good forecasting capabilities when utilized to predict the four data points which were 


withheld. This will be discussed in the next chapter. 


B. BOX-JENKINS METHOD 


The Box-Jenkins method of forecasting is comprised of three stages: 
identification, estimation, and forecasting [Ref. 9: p. 19]. In the identification stage, the 
series 1s often differenced to achieve stationarity about a mean (usually 0). During the 
development of these models, differencing of the order of one period (regular 
differencing) or four periods (seasonal differencing) was considered. As is standard 
with Box-Jenkins methodology, no more than two differencing corrections were 
heared ter any medel {Ref. 10: p. 125]. With one exception, a stationary series was 
achieved through the application of regular, seasonal, or a combination of both tvpes 
of differencing for each of the contractors. Contractor D data were already stationary 
and did not require differencing. The autocorrelation (ACF) and _ partial 
autocorrelation(PACF) functions of this stationary series are analvzed to determine the 
horizontal subpatterns within the series. The stationary trend can be specified as a 
linear combination of past series values (autoregressive terms), a linear combination of 
past random errors (moving average terms), or a combination of both. The 
determination of the appropriate number and specific lag of the autoregressive and 
moving average terms is made during the analysis of the ACF and PACF. Spikes on 
the ACF accompanied by trends on the PACF resembling exponential decav for the 
same lag indicate the appropriateness of a moving average parameter at tnat lag. 
Likewise, spikes on the PACF accompanied by an exponentially decaying trend on the 
ACF signal the need to include an autoregressive term at that lag. Spikes on the ACF 
and PACF are taken to be correlation values for a given lag which are statistically 


different from 0. Once the character of the trend is identified, the estimation phase 1s 





entered to determine the parameter = valies: 


TABIWEee 
SUMMARY OF CONTRACTOR REGRESSION Vio aie 


REGRESSION MODEL for COND RAGTC Rae 
Model: OVRHD(B) = at b PERSONNEL(B) 


Adjusted R° 927 
FoStatistic JA Sven 
Pmeeree gn. 11978.143 
Standard = rror, 59352.942 
Slope: ke2t5 
Standard Eimer: JTS 


REGRESSION MODEL for GON TRAG@i Ora 
Mode: OVRHD(@) = a+ bo PERSONNES) 


Adjusted R* $40 
Fo Stansmec 106.326 
lntercenr 327.038 
Standard Error: 3911.146 
Slope: 8.702 
Standard £161: 844 


REGRESSION MODEL for CONTRACTOR D 
Model: OVRHD(D) = a+ b PERSONNEL(D) 


Adjusted R- 726 
F Statistic 64.448 
[nterce pe 13864.996 
Standard Error: 3084.697 
Slope: 7.990 
Standard Error: 3G 


expressed as follows 


Z, = Oot OZ.) + Ay 
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A first order autoregressive series 1s 


a 
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—_—_= = — —_— - —_— 
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TABLE2 


Be vn Ole ON Pree PORGREGRESSION MIODELS (CONT'D.) 


| 
| 
| 


REGGE EoslOnetODEL fcr CONTRACTOR E 
WiccchmOnNmWtny — ar bo PERSONNEL(E) 


Adjusted R° 527 
Peo eatistic 26.590 
Neperee me, 4976.203 
Siandara Error. 18568.690 
Slope: 14.949 
Slancdarc Ee ipor-: 21099 


PGi Elo MODEL for CONTRACTOR F 
Moder sOvVRT Die) — at b PERSONS be F) 


Adjusted R- 707 
Bes eStic 63.739 
renee: 884.075 
Standard Error: 1083.769 
Slope: ON 
Stamaard Error: Ge? 


MEG Ne solo Ne ODED tor CONTRACTOR G 
Miodel@ny hii G)— at bePERSONNEL(G) 


Adjusted R7 857 
Statistic sso 
Paterce pt 4716.544 
Stancard Error: BO2OmS 2 
Slope: 20,545 
Standard Error: 1.780 


er a A 


a eee 


where 
Qo = series mean 
@,= weighting of the previous period value 


A,= white noise ~ NO,6 47). 


An alternative form of the model in terms of deviation from the series mean 1s 
where 


*, 


a srs Z,-9o- 
This expression can be expanded to include any number of past series values and is 
Written as follows: 


>” be wt 


Z, = 912.1 +9345 + = 0o2eeeee (eqn 3.11) 


* 
t-p 

The backward shift operator, B, is often utilized to express the relationship of the 
present term to the relevant past terms. The operator is a symbolic indicator and does 
not imply multiplication of Z, by a constant B. Its use indicates a desire to express 
past consecutive terms of a series as an ordered group. The operator possesses the 


following relationship: 


Bet ate Le 

B°Z, = 21.9 
m7 — 
Be 


Using the operator, the autoregressive process of order p 1s expressed as 


g(B)Z, =A, (eqn 3.13) 
with 
9(B) = (1-9, B-@5B°....- 9 BP). 


The moving average model assumes that the series can be expressed as a 
weighted average of past successive white noise terms. A first order moving average 
model expressing the series as deviation from the mean of the series 1s: 

Z. = AOA.) (eqn 3.14) 
where 
A, = series is white noise ~N(0,6 47) 
0, = coefficient of the most recent white noise term. 
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This basic relationship can be expanded to include any number of past terms. A 


moving average process of order q@ 1s written as: 


© = A,-8) A, -OrAy 9-8 GAL: (eqn 3.15) 


Z Ga t-q 


t 


Using the backward shift operator this model is expressed as 


Z, = OB)A, (eqn 3.16) 
with 
6(B) = 1-6,B-6,B--...-6,B4. 


A combination of these types of models can be developed and possesses 
relationships to past series terms and to past white noise elements. The process is 
called a mixed autoregressive-moving average model and is expressed as: 


7 * 


t= OZ tet O—Z py TAP BAL OAL g: (eqn 3.17) 


q tq 
This expression is usually written in terms of the backward shift operator, 


9(B)Z, = O(B)A,. (eqn 3.18) 


Models which adequately describe the data rarely exhibit values of p or q greater 
than two and usually are less than two. [Ref. 10: p. 66] 

The use of differencing to achieve a stationary series permits the use of the Box- 
Jenkins method to model series which are nonstationary in nature. The backward 


differencing operator V 1s used to indicate the following relationship: 
a eee le) (eqn 3.19) 


The model now becomes an autoregressive integrated moving average process 
(ARIMA) of order (p,d,q.) and is written: 


(B)V9Z, = (B)A,. (eqn 3.20) 


i) 


[In this relationship the superscript d indicates the number of times that regular, 
backward differencing was utlized to achieve a stationary series. 

Seasonal differencing was used in several of the models to achieve stationarity. 
The inclusion of seasonal differencing produces a seasonal ARIMA model described as 
(p.d,q) X(P,D.Q,),. The subscript s indicates the number of periods contained in one 
season. In the examination of quarterly data, there are four periods in one year and 
the subscript becomes a four. The general form of the seasonal ARIMA model is a 
multiplicative ARIMA model. The multiplicative nature of the model indicates that 
the miodel contains terms which are products of the regular and seasonal coefficients. 
Intuitively, this makes sense. In the case of quarterly data, the value in the series five 
periods previous to the present is included and has a coefficient which is the negative 
product of the first order term and the seasonal term [Ref. 10: p. 164]. A multiplicative 
ARIMA model of order (1,1,0}) X (1,1,0), is expressed as: 


ZL = QZ, .] +@,Z, 4-9); P)Z,_5 7A; (eqn 3-25) 
With 
AGE (Oana) e 
@, = coefficient for the seasonal term. 
The general expression for the multiplicative ARI MA model ts 
(B)®p(BvIv.2z, = 6 (B)O,(B)A 
Py Pees Sateen Oss 
(eqn 3.22) 


where 
OQ = coefficient of the seasonal error term 


Pp = coefficient for the seasonal term. 


Several plausible models are developed during the estimation stage and these 
alternative models can be compared utilizing the diagnostics provided in most 
computer packages. The parameters and the associated residuals from each plausible 
model were examined to validate the model. The residuals were examined for the 
presence of bias and autocorrelation. The three indicators used to detect the presemesc 
of these conditions were the residual mean and variance, the autocorrelations of the 


residuals, and the Q statistic. The model parameters were examined for statistical 
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significance and indications of high correlation between each other. Highly correlated 
parameters are usually an indication of the inclusion of unnecessary parameters in the 
model [Ref. 10: p. 98]. The desired outcome is to determine one or more models which 
produces fitted values as close as possible to the original series. Additionally, it is 
desired that the models have as few parameters as possible [Ref. 9: p. 17]. The final 
stage of the Box-Jenkins method, forecasting, enables the user to project the series into 
the future. Often. 95% confidence intervals for the projected values are provided by 
the computer package. The computer package utilized during this analysis was 
GRAFSTAT. It is a package being developed by IBM and is installed at the Naval 
Postgraduate School for evaluation. 

In general, the results derived from the Box-Jenkins method are usually more 
accurate in the short and intermediate term than forecasts from other methods, 
including regression [Ref. 11: p. 236]. The cost of these generally superior results are 
measured in the computer resources required to derive the model and the expertise 
required of the statistician to determine an appropriate model. The procedure allows 
for interpretation on the part of the forecaster. Two forecasters mav identify different 
models as being the best model to fit the same data set. Even so, both sets of forecasts 
mav be highly accurate when compared to the future observations of the series 
mevey. 3: p. 11]. 

The Box-Jenkins models developed for each of the contractors displaved strong 
predictive properties when used to predict the missing values. Table 3 presents the 
differencing required to achieve stationarity and the final form of the model for each of 
the contractors. The model for each contractor is expressed in standard Box-Jenkins 
notation with a seasonal period of four. The coefficient values, the model mean, and 


the standard errors of these terms are included in the presentation. 


C. BOX-JENKINS TRANSFER FUNCTION 

The Box-Jenkins transfer function is a procedure which allows a forecaster to 
aggregate the information contained in a particular series (output) with one or more 
related series (input) to forecast future values of the series. The relationship which is 
MsWaly identified is that the trend present in the input series is reflected in the output 
series after a lag of several periods. Relationships of this order are referred to as 


dynamic responses. The aggregation of information is achieved through a transfer 


TAREE. 


SUMMARY OF BOX-JENKINS ViIGB Es 





Summary For Contractor A 


ARIMA (1300) (O,1:0.)c 

AR(1) joe 
Standard Error old 

Viean 2023-270 


Stalicendee mon 6648.622 


Summary For Contractor B 


ARIMA (1,1,0) x (0.1,0,)4 

AR(1) -.845 
Standard Error re 

Vlean 488.515 


Standard Error bl62.382 


Summary For Contractor C 


ARIMA (12,0) Geo 

AR(1) -.673 
Standard serror alice 

SAR(1) .764 
Standard Error 103 

Mean 305.319 


Standard Error 416.400 


Summarv For Contractor D 


ARIMA (1,0;0)5= (0,0,0,)4 

AR(1) .688 
Standard Ermor os 

Mean 189328.647 


Standard Error 6599.820 
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Summary For Contractor E 


ARIMA ciel (0,0,0)4 

MA(1) 906 
Standard Error O59 

\fean HSS2 20, 
Standard Error 897 G37 


| 
Summary For Contractor F 
| 


AR(1) -.607 

, Standard Error .166 

| SAR(1) STD 

| Stamens Error lee 
Mean -350.716 


| 
ARIMA (1,1.0) X (1.1,0.)4 


Standanca Error P53 457 3 | 


ARIMA (iO Oh (0,1,0)4 

AR(1) 854 
Standard Error mS 

Mean fi 25;453 


| Sunimarv For Contractor G 


Standard Error 18744.529 


Mimecion miodel [Ref, 9p, 13)” This procedure is comprised of the same three phases as 
the univariate Box-Jenkins method. During the identification stage, differencing of the 
Series 1s usually recommended in an effort to achieve a stationary series. However, a 
linear combination of elements in the output series often may be stationary, and 
differencing of all of the series in the model can cause complications in identifying the 
appropriate model [Ref. 12]. 

The data trends may be tentatively identified and prewhitened in an effort to 


achieve an input series which strongly resembles white noise. This effort 1s made in an 


attempt to improve the interpretability of the cross correlation function (CCF). If the . 
input series 1s not prewhitened, the CCF often cannot be interpreted [Ref. 13: p. 243]. ( 


A series which can be represented by an ARIMA model 
xX, = (1-9, B-...-9,,BP) (1-8 B-...-8 BA, (eqn 3.23) 
Or as expressed with the backward shift operator 
X,= 0, !(B)0,(B)A,.- (eqn 3.24) 


can be prewhitened by inverting the model. The prewhitened version of the series is 


expressed as follows: 
a. = 9, (B)6,! (B)X,. (eqn 3izen 


The same transformation is applied to the output series culminating in the following 


expression: 
B, = ©,(B)0,"! (B)Z,. (eqn 3.26) 


The relationship between the input and output series is described in an impulse 


response function of the form 


tJ 
~~] 


Ly = VQX, TVA, | TV JXr9 +. EM, (eqn se ) 
where 
v is the impulse response 


N, 18 the random noise term. 


Expressing the impulse response function in terms of the prewhitened series, @, and B,, 


the function becomes 


Be > UGs eee (eqn 3.28) 


t Ie 


Where €, is the transformed noise series defined by 


€. = ,(B)0,"! (B)n, . (eqn 3.29) 


The impulse response Weights can be determined through examination of the 


CCF. The following relationship 1s utilized in this regard: 


Pa BiK)SRp 
——_ hea ieee eqn 3,90} 
o 
where 
Pa Bis) = correlation coetficient between the 


d, and B. series for the ,th element 
Sq = estimated standard deviation of the d. serles 


SB = estimated standard deviation of the B. series. 


Once estimates of vb; are obtained, identification of those elements of the impulse 
function which are statistically significant enables the forecaster to identify an 
appropriate model to be used as a basis for the transfer function. From these 
estimates, simultaneous equations can be formed and solved to provide preliminary 
estimates of the parameters of the model. Since the impulse responses provided by the 
(Cr are Statistically ineflicient im@general, the proposed model is used as a starting 
point to be fitted by some more elaborate means. 

This procedure 1s extremely complicated and places a high demand on computer 
resources and the skills of the forecaster. The results should be indicative of the cost of 
attaining them. I[n this particular application of the procedure, several difficulties were 
encountered. A computer package to perform the entire procedure was not available. 
The GRAFSTAT package mentioned previously does contain a CCF routine. 
Therefore the inefficient estimates of the impulse responses obtained from the 
examination of the CCF could be used to form simultaneous equations. However, this 
would be the most elaborate means of determining the parameters. The data from 
each contractor were examined in all combinations of undifferenced. regularly 
differenced, and seasonally differenced forms. In all cases, the data were prewhitened 
before eXamination. Only one CCF indicated the presence of an impulse response 
value which was statistically different from zero. This was the prewhitened and 


imeurercmeed series for Contractor D. Therefore. the only input value used in the 


forecast of the output was the present input value. The nature of this model led to a 
single equation of the following form: ac 


Vp = 14.39 = ao. (eqn 3.31) 


From equation 3.31, the final multivariate Box-Jenkins model was determined to 
De: 


Y, = 14.39 X,. (eqn 3.32) 


The model parameters are listed in Table 4. The forecasts made with this model are 
inferior to those obtained from the univariate Box-Jenkins model. As will be discussed 
in the results chapter, the presence of the input series (X,) does not produce a 
regression model or a transfer function which outperforms the univariate Box-Jenkins 
model for contractor D. The inability of the procedure to develop adequate models for 
the majority of the contractors mav be attributable to the relatively small sample size 
of each data set. The longest series available for analysis was twenty-seven 
observations and the majority of the contractors supplied data in the range of twenty 
to twenty-four observations. Generally, sample sizes larger than sixty observations or 
data spanning at least eight complete seasonal periods are recommended as a minimum 


number of data points for the method to perform well [Ref. 3: p. 6]. 


TABLE 4 
SUMMARY FOR CONTRACTOR D 


| Pap AI 
Sq 1034.734 
Sp 30387.40016 


IV. PREDICTION METHODOLOGY AND RESULTS 


Each model was used to predict the four withheld data points. For the GLS 
models actual X’s for the four periods were available and used. In those cases where 
only one form of autocorrelation was present in the data, a transformation of the 


following type was made to the original data prior to estimation: 


MONA ne , 
Ys = p¥ eS p; re) (eqn 4.1) 
(t= T-3,T-2.T-1,T) 
GQ=1or4). 


The majoritv of the GLS model required two transformations to remove all forms of 
autocorrelation. In those cases, the X’s and Y’s were transformed in the following 


manner before the final GLS model was used to predict the withheld values. 


Transform the data for AR(4): 


*k s: JN 
an zs YyPayt4 
Xt al Xe Pary.g (eqn 4.2) 
(tes, 0n ace, 


Transform these new data to remove the effects of AR(1): 


2 Ny 72 
me = Y re Py Y t-1 
r ee a TX _ 4% 
Ny =X reP] dM t-] (eqn 4.3) 
(alee ee aii 


Substitution of equation 4.2 into equation 4.3 allows both equations to be 


combined as follows: 


ee tele oN as “NS /~ 
Xy = Xe PX g7P (Ay P45) (eqn 4.4) 


(t=6,7,8,...,T) . 


yi 


These equations can be simplified to the following form: 


No ares ae YN 1» oe 
Yo = Ye Ppl ep Pale4 F Py PaYe.s 


tc ~~ ws “SN fs - 
Np = ANy Py Xe PaXy.4 7 Pp Py Se.5 (eqn 4.5) 
(t= 6.7 3.) 


Assuming that all forms of autocorrelation have been removed, the following 
relationship holds: 
4c ok oe ie 
eae (eqn 4.6) 
v, ~ N(0,67) 
(C16, S550, 


Y 


The entire transformation can be made in one step and forecasts can be made 


with the following equation. 


eA AA. 
Yor Pp Yep Tr Pg veg PpParegt 


“~~ fn 
(XP Xp. [PyXt.4°P] PgXt.5)B (eqn 4.7) 
(t= 6,7,8,...,T) . 


This set of equations pertained to the instance in which AR(4) was removed first and 
AR(1l) was removed during the second application of GLS. These equations were 
Geveloped by substituting the transformation for AR(4) into the equation for the 
removal of AR(1) processes. A similar expression was derived for those cases in which 
the correction for AR(1) preceded the removal of AR(4) processes. In the case of the 
Y’s, this procedure required an iterative process to determine the last three values in 
they Vector, 

The Box-Jenkins models developed for each contractor were used to produce 
forecasts which were compared to the withheld data points. These forecast values were 
provided by the GRAFSTAT package. The package also provided ninety-five percent 
prediction intervals for each forecast. A graphical presentation of the forecast 1s 
provided with an analysis of each contractor’s data in the remainder of this chapter. 

Forecasts were made with the multivariate Box-Jenkins model. These were made 
by substituting the known values in the input (Xt) series into equation 3.32 and 


completing the computations. 
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The forecasting capabilities of each model were measured by three comparison 
indicators: correlation coefficient between the actual and predicted values, root mean 
square divided by the mean of the actuals. and the mean absolute percentage error. 
An analvsis of the data and a presentation of the prediction results for each contractor 


follows. 


A. CONTRACTOR A 

Contractor A supplied twenty-two data points. The data was categorized and 
deflated to constant dollars as specified in Chapter If. Graphical presentations of the 
raw data 1s presented in the top two graphs of Figure 4.1. The upper left hand graph 
1S a presentation of the overhead cost across time (twenty-two consecutive quarters). 
Likewise, the upper right hand graph 1s a display of the direct personnel trend across 
tume. The overhead cost versus time graph displays a sharp decline in the first four 
quarters and a somewhat cvclic increase thereafter. The trend is increasing in general. 
The direct personnel versus time graph indicates a similar trend in general, but does 
not appear to be influenced by a seasonal trend to the extent that the overhead cost 
trend is. The lower left hand graph displays the relationship of overhead cost versus 
direct personnel. The weak relationship is depicted graphically and in the results of the 
OLS regression (adjusted R? = .183). The lower right hand graph in Figure 4.1 shows 
the relationship of overhead cost to direct personnel after both series have been 
fiamsiormed to remove the effects of autocorrelation. The adjusted R* for this model 
is .8/87 as indicated in Table 3. The graphical portrayal of the transformed data 
suggests that the GLS model should be a dramatic improvement over the OLS model 
Pea prediction tool, A summadzy of the predictive results of the GLS model Is 
presented in Table 5. Actual X’s were available and used in making predictions for the 
four withheld quarters. 

The Box-Jenkins model developed for Contractor A was presented in Chapter 3. 
A topic of concern was the amount of data available which could be used as a basis for 
the model. Generally, the amount of data needed to develop accurate models 1s fiftv 
observations with one hundred preferred. Models can be developed in the absence of 
these amounts of data, but the forecaster must utilize experience and past information 
to develop preliminary models which can be updated as more information becomes 
available [Ref. 9: p. 18]. A graphical portrayal of the results of the Box-Jenkins model 


is presented in Figure 4.2. The left hand graph is the actual overhead series including 
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Regression Analysis Graphs for Contractor A. 
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Figure 4.2 Box-Jenkins Graphs for Contractor A. 





EAE ies 
PREDICTION RESULTS FOR CONTRAGHGERS 


Mean Absolute Percentage 


BOX-. 
REGRESSION JENK ISS 
| Correlation Coefficient: B62 ol 
| Root Mean Squared Error 
| Mean of the Actuals: 0649 0376 
| 


the periods for which predictions were made. The right hand graph is the first twenty- 
two observations and the four prediction values. The trend of the predicted values can 
be compared to the actual trend by mentallv superimposing one graph on the other. 
As indicated in Table 5, the results of the Box-Jenkins model for Contractor A 1s 
Superior to the resuits of the GLS model. 

Attempts to develop a multivariate Box-Jenkins model were made and proved to 
be unsuccessful. The cross autocorrelation function (CCF) was plotted. However, 
none of the impulse weights were statistically significant. All reasonable combinations 
of differencing were examined in addition to a CCF in which no differencing was 


included. The outcome was the same in all cases. 


B. CONTRACTOR B 

Figures 4.3 and 4.4 and Table 6 are provided for Contractor B. Figure 4.3 
displays the raw overhead cost and direct personnel series for Contractor B. As 
indicated in the upper right hand graph, the overhead series has an increasing trend 
accompanied by a seasonal variation. The direct personnel series is characterized as a 
consistently increasing series. A comparison of the slopes of the series in each graph 
indicates that the quarterly direct personnel count is increasing at a slightly greater rate 
than the overhead cost per period is. The overhead cost versus direct personnel chart 
reveals the presence of a relationship between those two series which is stronger than 


the near randomness revealed in the same graph for Contractor A. This is supported 
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Figure 4.4 Box-Jenkins Graphs for Contractor B. 
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Pee Dion Reset hs FOR CONTRACTOR B 


BON. 
REGRESSION JENKINS 


! Correlation Coefficient: 584 675 

| 

! Root Mean Squared Error; 
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| Mean Absolute Percentage 

| ERROR, 8.58 3.91 
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a 
by the results of the OLS model which are surprisingly good with an adjusted R? of 
669. After transformation, the GLS model achieves an adjusted R* of .927. There are 
three transformed observations which are easily distinguished from the remainder of 
the transformed data. These observations are located closely together on the 
transformed data graph. The GLS model developed possesses all of the indications of 
a statistically significant and worthwhile prediction model. The adjusted R-, F- 
statistic, and T-statistic for the slope are all significant [Ref. 14: p. 133]. Despite this 
fact, the Box-Jenkins model developed for Contractor B is superior as a forecasting 
tool in the range which is being examined in this paper. The multivariate Box-Jenkins 
model suffered from the same short comings as the model for Contractor A. The 


impulse weights were determined to be statistically insignificant. 


C. CONTRACTOR C 


Figures 4.5 and 4.6 and Table 7 are provided for Contractor C. The overhead 
cost and direct personnel graphs derived from Contractor C data are presented in 
Figure 4.5. The overhead cost trend appears to fluctuate significantly about a mean 
Value of approximately $90,000,000 during the first seventeen quarters with the 
minimum value of the series occurring in the fifteenth quarter. The series shows a 
departure from this trend during the last eight quarters. The last four values depicted 
on the top two graphs are values which were not included during the model 


development stage. These are the values which are being withheld for prediction 
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Figure 4.6 Box-Jenkins Graphs for Contractor C. 
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comparison purposes. The direct personnel graph shows an increase during the first 
nine periods followed by a decrease for approximatelv eight periods. At that point, the 
number of direct workers emploved at Contractor C increases for the remaining eight 
periods. This trend matches the overhead cost trend in general for the entire length of 
the data strings, but does not appear to be influenced by a seasonal component. The 
plot of overhead cost versus direct personnel appears nearly random (adjusted R? = 
291). The plot of the transformed data displays a strong direct relationship between 
the two variables. As the number of personnel increases, the overhead cost increases. 
This is apparent in the GLS model which has an adjusted R* of .840 and an F statistic 
of 106.326. The Box-Jenkins model graphs are presented in Figure 4.6. These graphs 
are difficult to mentally superimpose on each other. This problem ts caused by the 
bounds of the 95% confidence interval for the forecast observations. In the case of 
Contractor C, the regression model produces predictions which are actually closer to 
the actual values than those calculated by the Box-Jenkins model. A comparison of 
the predictive results is presented in Table 7. The multivariate Box-Jenkins model was 


unusable for Contractor C. 


D. CONTRACTOR D 
The data supplied by Contractor D spanned twenty periods and is presented in 


Figures 4.7 and 4.8 and Table 8. The overhead cost versus time plot indicates that the 
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Figure 4.7. Regression Analvsis Graphs for Contractor D. 
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Figure 4.8 Box-Jenkins Graphs for Contractor D. 
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ee ee 
series started at a relatively low point and increased rapidly in the fifth quarter. The 
trend remained relatively constant for six periods at which time it began to decrease 
slowly. This trend remained consistent through the twentieth quarter which Is the last 
period included in the model development stage. The direct personnel series is similar 
Wemtnie Overhead cost trend with an abrupt decrease in the thirteenth quarter. 
Beginning in the twentv-first quarter, the direct personnel trend becomes flat and 
neither increases nor decreases for the remainder of the series. These last four values 
are the known X's which are used in the regression model and the multivariate Box- 
Jenkins models. As the X’s (direct personnel) become a level function in the prediction 
interval of the data, the Y’s (overhead cost) suddenly increases. This departure from 
the previous trend causes problems with both of the models which utilize the available 
Pee ailics to Generate predictions. The effect of this departure from the trend is 
Mmetcated by the relatively poor results of these models as listed in Table 8. The 
multivariate Box-Jenkins model is predicated on a single impulse weight which was 
significant. This occurred at a lag of zero periods. The model which was developed 
utilized only the present period X value to predict the value of Y. Therefore, this 


model was susceptible to the trend departures present in the data. 


E. CONTRACTOR E 
Figures 4.9 and 4.10 and Table 9 are provided for Contractor E. The regression 
model developed for Contractor E is unique in the fact that it did not need to be 


corrected for autocorrelation. As indicated in the graphs in Figure 4.9, both the 
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Figure 4.9 Regression Analysis Graphs for Contractor E. 
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Figure 4.10 Box-Jenkins Graphs for Contractor E. 
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REGRESSION JENKINS 

Correlation Coefficient: -,434 -.606 

: Root Mean Squared Error’ 

| Mean of the Actuals: 107, .137 


\fean Absolute Percentage 
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overhead cost and the direct personnel series are generally increasing with time. The 
overnead cost series tends to fluctuate above and below the general trend throughout 
the length of the series with the most significant departure from the general trend 
occurring in the twenty-first to the twenty-fifth data observations. The direct 
personnel series increased initially in the first ten periods and remained fairly constant 
at a level of approximately 6500 workers for the next ten periods. This senes shows a 
sharp increase in the twenty-first period and continues to increase at a somewhat 
slower rate thereafter. The overhead cost versus direct personnel graph displavs a 
direct relationship between these two variables in the absence of an autocorrelation 
correction. The strength of this relationship is less apparent as the number of 
personnel increases. The four observations in the upper nght hand corner of this 
graph are the last four observations in each series in the model formulation range. 
Their proximity to each other is a function of the fact that the direct personnel series 1s 
slowly increasing in the twenty-first through the twenty-fourth observations. This 
explains the remoteness of their placement on the graph as they occur later in time 
than the large increase in the number of workers that was recorded in the twenty-first 
quarter. The appearance of these points as a nearly vertical line is a function of the 
smal! increase in the direct personnel component of the graph and the large 
luctuations that occurred in the overhead cost series in the twenty-first to twenty- 
fourth quarters. The OLS model has an R* of .527 and an F statistic of 26.59. 
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The ARIMA model for Contractor E was also unique in the fact that it was the 
only contractor model which utilized a moving average (MA) process to describe the 
data trend. This model was difficult to identify and was actually determined through a 
nrocess of elimination. Every reasonable model was analyzed and the ARIMA 
(O.1,1) x (0,0,0) ; was chosen on the basis of the statistical significance of the coefficient 
and the model’s performance as a forecasting tool. Several models which actually 
produced slightly better prediction results were excluded from consideration because 
the resulting coefficients were not significant. The multivariate Box-Jenkins model 
could not be developed. The prediction results of the regression and Box-Jenkins 


ma@dels are presented in lable 9 . 


F. CONTRACTOR F 


Contractor F was the only contractor to supplv data covering the entire 
observation period. Figures 4.11 and 4.12 and Table 10 are provided for Contractor F. 
The overhead cost and direct personnel series display similar trends. Both series 
increase until the ninth period, then decrease in general and eventually resume 
iiereasing.. Phe overhead cost trend resumes increasing in the sixteenth quarter. This 
latter trend in the overhead cost series displays a seasonal fluctuation with the 
generally increasing trend. The direct personnel series decreases until the sixteenth 
period at which point it becomes a nearly constant function for six quarters. Beginning 
in the twenty-fourth quarter, the number of workers begins a rapidly increasing trend 
and continues in this manner through the remainder of the series. Several significant 
Pcie increases and decreases occur within the direct personnel trend. Three of these 
rapid changes in the number of direct workers employed by Contractor F border above 
and below the 3600 to 3800 interval in the work force level. The absence of data 
observations in this range creates two distinct clusters of observations in the overhead 
cost versus direct personnel graph. This plot does not appear random because of the 
blank interval between the two groupings. However, the plot does not display a strong 
relationship between the components either. The OLS model has an R? of .317. Both 
forms of autocorrelation, AR(1) and AR(4), were removed from the raw data through 
transformation and the resulting series displays a direct relationship as depicted in the 
lower right hand graph of Figure 4.11 The application of GLS to the raw data 


improves the model significantly. 
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Regression Analysis Graphs for Contractor F. 


Figure 4.11 
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Figure 4.12 Box-Jenkins Graphs for Contractor F. 
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TABLE 10 
PREDICTION RESULTS FOR CON PREGie@ ra 


BOX-, 
| REGRESSION JENS 
| Correlation Coefficient: ozs 440 
| Root Mean Squared Error; 
Mean of the Actuals: 08 86 0823 
Mean Absolute Percentage 
ETEOr: 8.23 6.86 


The Box-Jenkins model appears to approximate the trend well but underestimates 
the magnitude of the actual values. This model is approximately equal to the GLS 
model in prediction power. The attempted development of the multivariate Box- 
Jenkins model for Contractor F failed to indicate a significant impulse weight. 


Therefore, the development of the model was not pursued. 


G. CONTRACTOR G 


The data supplied by Contractor G spanned twenty-four quarters. The graphs in 
Figure 4.13 indicate that both the overhead cost and direct personnel series follow 
similar patterns during the twenty-four quarters. There is an increasing trend which 
becomes a decreasing trend in the vicinity of the seventh quarter for both series. The 
overhead cost series displavs the influence of seasonal fluctuations about the general 
trend. The lower left hand plot of overhead cost versus direct personnel indicates that 
the uncorrected data possess a noticable direct relationship to each other. The R? 
value for the OLS model was .541. After the data were corrected for both AR(1) and 
AR(4) processes resulting in a GLS model with an R? of .862. The F statistic for this 
model was 133.155. Table 11 illustrates the outstanding predictive results of this 
model. The Box-Jenkins model for Contractor G is portrayed graphically in Figure 
4.14. The scaling of the two graphs is different because of the inclusion of the ninety- 


five percent confidence intervals in the forecast graph. As indicated in Table II, the 
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.13. Regression Analysis Graphs for Contractor G. 
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Figure 4.14 Box-Jenkins Graphs for Contractor G. 
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QUARTERS 


QUARTERS 


TABLE 11 | 
PREDICTION RESULTS FOR CONTRACTOR G | 
| | 
| | 
| | 


BOX-. 
REGRESSION JENKINS | 
| 


Correlation Coefficient: .638 .604 
Root Mean Squared Error’ 
Mean of the Actuals: (Oo! 0394 
Mean Absolute Percentage 
! ecorg 1.96 Seo 


model is an effective forecasting tool for the specified prediction range. However, the 
regression model is superior in all three categories of comparison. The multivariate 
Box-Jenkins model could not be developed for Contractor G. The recurring problem 


of insignificant impulse weights plagued this data set as it has several others examined 


in this thesis. 


V. CONCLUSIONS 


The intent of this project was to develop and compare forecasting models to be 
used in the prediction of overhead costs for seven government aerospace contractors. 
This 1s part of a continuing overhead tracking project at the Naval Air Svstems 
Command. Three types of methodology were considered as possible model sources. 
These included least squares regression models, Box-Jenkins methods, and Box-Jenkins 
transfer functions. 

The regression models which were examined were developed to be used =by 
unsophisticated forecasters and operate well on a microcomputer. The characteristics 
of the data suggested the complications in model development which are associated 
With autocorrelation. Consequently, these models test for two forms of autoregressive 
processes: AR(1) and AR(4). Any necessary adjustments are made within the model. 
A review of several literature sources indicated that the predictive capabilities of 
regression models are usually inferior to those of the Box-Jenkins method in the short 
run. In general this was found to be true. However, the regression models performed 
well in most cases and resulted in superior forecasting models for two of the 
contractors. 

The Box-Jenkins class of models 1s considerably more complex than least squares 
regression. Therefore, a more experienced forecaster and more efficient compmies 
resources are required to employ this method. The majoritv of the Box-Jenkins 
transfer function models could not be developed for the data in this project. The 
difficulties encountered are probably due to the small sample size for each data set. 
Additionally, a computer package was not available which could perform the entire 
transfer function procedure. 

In those cases in which these requirements could be fulfilled, ether of the Box- 
Jenkins methods should be utilized. In the absence of continuous access to such 
capabilities such as is the case in the Naval Air Systems Command overhead tracking 
project. least squares regression theory can be used to develop very adequate models 
which produce results which are usually far superior to the results obtained from the 
prevailing practice of applying estimated overhead rates to estimated direct labor 


hours. 
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APPENDIX 
APL FUNCTIONS 


itemrolOVmeewr eeimetions periorm the OLS and GLS regressions. The GLS 


function transforms the data to remove AR(4) processes and GLSD makes the required 


transformation for AR(1). The PRED function computes the prediction effectiveness 


measurements for the OLS, Box-Jenkins. and transfer function forecasts. The PRED1. 


PRED4., and PRED4O functions perform the same task for the regression models which 


fequinecasamtransionmmation due to the presence of autocorrelation. The PREDI 


function was utilized in those cases where only AR(1) processes were detected during 


the regression model development stage. The PRED4 function performs a similar task 


for those cases in which AR(4) processes are the only form of autocorrelation. The 


PRED4O function makes the appropnate transformations for both AR(1) and AR(4) 


processes regardless of the order in which they were removed. 
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